a) Field of the Invention
This invention relates to a biometric machine method for quantitative assessment of fine motor control of human individuals through the monitoring of changes in cursive writing dynamics. Such quantitative assessment uses Correlation Function Analysis of handwriting dynamic signals, such as acceleration and pressure, in time domain, and returns values of criteria for stability, smoothness and synchronization of handwriting motions.
b) Description of the Prior Art
It has been very difficult to identify individuals with unset Parkinson""s and other neurological diseases because of subtlety of the early symptoms It is therefore difficult to identify and adequately characterize the unset of fine motor control deficiencies in such individuals. It has also been difficult to assess impairment or impediment in fine motor control due to the influence of alcohol and drug substances, or from environmentally caused distress. Handwriting is a complex cognitive motor skill. Since handwriting is usually a well-learned skill that is generally used on a daily basis, the motor behavioral aspects of handwriting are theoretically interesting and practically important particularly in the early identification and assessment of fine motor control impairment as a result of the above-mentioned problem. Thus, research based on the changes in handwriting dynamics have been considered in an effort to better understand, identify, and assess human fine motor control. In the recent past there has been a growing interest in research in fine motor control of humans through analysis of handwriting dynamics. Several publications in the field of Neurology represent this trend. For example, David Margolin and Alan Wing, in their paper xe2x80x9cAgraphia and Micrographia: Clinical Manifestations of Motor Programming and Performance Disordersxe2x80x9d published in Acta Psychologica, 1983, presented the results of research of acquired disorders of handwriting. They compared acquired agraphia due to cerebrovascular accident to micrographia produced by the Parkinsonian patients. They stated, xe2x80x9cexisting reports of apraxic agraphia . . . do not provide much information about handwriting control from an information processing standpoint, although they can be useful in determining the anatomical localization of handwriting control. Overall, the reported cases of apraxic agraphia indicate that handwriting can be disrupted without affecting other motor skills.xe2x80x9d Parkinsonism, on the contrary, xe2x80x9caffects all voluntary movements, but handwriting appears to be particularly vulnerable, in that it is frequently the first manifestation of this disease.xe2x80x9d And they conclude xe2x80x9cif handwriting is indeed sensitive to disturbances of the extrapyramidal motor system then a quantitative analysis of writing could serve as a useful tool in evaluating diseases which affect this system and provide insights into dynamics behind these handwriting changes.xe2x80x9d The present invention provides such new channels of information about motor control of handwriting and will shed light on the complex mechanisms of fine motor control.
In the research of handwriting generation by R. Plamondon published in Acta Psychologica 82 (1993), the author states xe2x80x9cstrokes must be superimposed to generate fluent handwriting. This is in accordance with a basic psychophysical phenomenon often reported in motor control: the handwriting generation process, like many other types of movements, is not exclusively sequential, and very often advanced preparation of the forthcoming stroke is done in parallel with the execution of the actual stroke. . . . In other words, the basic strokes are hidden in the signal.xe2x80x9d In conclusion, Plamondon suggests the method to extract strokes by performing xe2x80x9can analysis-by-synthesis experiment, with the help of the proper impulse response for each strokexe2x80x9d and the use of quite complicated models of reconstructing the curvilinear velocity profiles defined by a log-normal equations. However, Plamondon presents a method for mathematical modeling of individual strokes and fails to teach a system for analysis of sequences of strokes. The extraction of single strikes and their modeling and analysis, as proposed by Plamondon, would result in loosing important information about behavior of strokes in their sequence. In addition to that, identification and high precision modeling of individual strokes represents very difficult and even unfeasible task because of greatest complexity of each handwriting movement where numerous elements of the central control and neuromuscular systems are involved in their performance. The present invention teaches a new analysis of the handwriting dynamic signals as indivisible collections of strokes intermittent with time distortions, contrary to considering individual strokes.
E. Parkins in his article xe2x80x9cCerebellum and Cerebrum in Adaptive Control and Cognition: a Reviewxe2x80x9d published in Biological Cybernetics, 77 (1997), explores the relative roles of the cerebrum and cerebellum in adaptive control. Parkins makes an interesting observation: xe2x80x9cVoluntary movements may first be performed and controlled by relying on feedback from sensory organs, but after some practice the same movement will be performed without feedback, the movement being performed more quickly and more automatically with less conscious effort. Here, practice converts the mode of voluntary movement from feedback to feed forward. The cerebellum operating as an adaptive feedforward system may be inserted in parallel to the cerebral cortex operating as an executive feedback-control system.xe2x80x9d The present invention, through measurement and assessment of handwriting dynamics representing both automatic, such as signature, and cognitively controlled cursive writing, such as xe2x80x9cllllxe2x80x9d, xe2x80x9clmlmxe2x80x9d, xe2x80x9clelexe2x80x9d, etc., provides valuable and previously unavailable information about the role and interplay of cerebellum and cerebrum in adaptive motor control and cognition.
Between 1994 and 1997, a joint European project, MIAMI(Multimodal Integration for Advanced Multimedia Interfaces) for the universities of The Netherlands, Germany, Denmark, and Italy was conducted for extensive research of human handwriting. The report of MIAMI resents the following observation: xe2x80x9cContrary to speech, cursive handwriting is not an innate neural function, and must be trained over several years. During the training process, handwriting evolves from a slow feedback process involving active attention and eye-hand coordination to a fast automatic and ballistic process. The atomic movement unit in cursive handwriting is a stroke, which is a movement trajectory bounded by two points of high curvature and a corresponding dip in the tangential movement velocity. The typical modal stroke duration is of the order of 100 msec, and varies less for increased movement amplitudes than one would expect. For a large range, writers exert an increased force in order to maintain a preferred rhythm of movement. Once xe2x80x9cfiredxe2x80x9d cortically, such a stroke cannot be corrected by visual feedbackxe2x80x9d.
Some early publications in the field of physiology also represent interest for the present invention. In 1864, Russian physiologist K. I. Barr published his work in which he first introduced the concept of xe2x80x9cbiological quantum of timexe2x80x9d which was further developed by German researchers J. V. Uexkull and G. Kriszat, who called it xe2x80x9cphysiological momentxe2x80x9d in their paper xe2x80x9cStreifzuge durch die Umwelten von Tieren und Menschenxe2x80x9d, 1970. It is very plausible to assume that the stroke, the atomic movement unit in cursive handwriting, is a spatial representation of the biological quantum. The biological quantum of time is estimated to be 100 to 180 msec. In addition to the literature, several devices have been developed to measure tremor. The primary known devices include:
1. Potentiometers to indicate motion of an extremity. They use a mechanical linkage, similar to an articulated dentists"" drill arm with a potentiometer at each joint. However, such linkages very greatly restrict the physical motion of the extremity, and can create backlash and hysteresis.
2. Electromyographic (EMG) surface electrodes attached to the skin are used to . . . assess tremor in the muscles during cursive writing. The problem with EMG is that signals that are indicative of tremor are obscured by remote muscle activity and electrical interference. In addition, the EMG equipment is cumbersome, not easily manageable and not portable.
3. Accelerometers to record acceleration of the upper extremity movement, and to compute the velocity of the extremity as its first integration and the displacement as its second integration. Accelerometers are mounted on a Lucite board strapped to the hand that is supported by a second board, forming a sandwich with the hand between the boards. In the implementation used, one accelerometer could not distinguish the actual tremor from the perceived motion, since a slight reorientation of the sensitive axis of the accelerometer with respect to the earth""s gravitation field can cause it to record an acceleration variation without any actual tremor. Due to any orientation shifts and bias error, which represents the non-zero output when there is no input, just the first integration of the accelerometer""s output to obtain the velocity of the motion would generate ample error to make use of accelerometers for measurement of motion impractical.
4. Gyroscopes sense the angular rate as a function of time and frequency. By utilizing one or more small solid-state gyroscopes, which are attached to hands, feet, or head, physical motion can be quantified. However, the gyroscope system is limited to measuring the angle variations of motion only, and like all previously mentioned devices requires attachment to the subject""s body.
5. Applicants, Alexander Livshitz, now known as Alexander Landau, and Ruth Shraiman have been granted U.S. Pat. No. 5,202,930 for High Precision On-Line Dynamic Signature Verification System. It teaches a machine method for electronic on-line signature verification based on comparison of the dynamics of sample and reference data. Multi-dimensional cross-correlation function analysis is applied, with use of sliding window mechanism for mapping phase coincident regions of the reference and to-be-verified signals. By establishing high level of statistical similarity between the said signals, the method provides means for authentication of an individual in a variety of security applications, such as computer networks access, credit card authentication, electronic voting, and the like.
However, none of these systems or literature teach a system for a rapid, non-intrusive, reliable, and low-cost system for quantifying fine neuromuscular performance and fine motor control skills of human individuals using cursive writing and minimal equipment.
It is therefore an object of the present invention to provide a rapid, reliable, non-intrusive, and low-cost system for quantifying fine neuromuscular performance and fine motor control skills of human individuals using cursive writing and minimal equipment.
In addition, it is an object of the present invention to provide a method for instantaneous, within the subject evaluation of fine motor control, without the need to have reference data for that individual.
The present invention addresses the problem of quantifying assessment fine motor control skills in human individuals. As detailed below, the method applies correlation function analysis for instantaneous, within the subject comparison of behavioral random signals of cursive handwriting dynamics within the human individual, without the need to have reference handwriting dynamic data from that individual, and use that data to derive the measures of stability, smoothness and synchronization of the hand motions. Those measures of hand motion are compared against the data values established across the healthy population of humans, as represented by their mean and standard deviation, and using the comparative data to assess any deterioration in fine motor performance of an individual. This is accomplished,. as taught by the present invention by providing a method, apparatus and machine method for instantaneous assessment of fine motor control of humans through analysis of cursive handwriting dynamics by using the application of correlation analysis for comparison of random signals that are represented by repetitively produced cursive handwriting samples by an individual.. In the practice of the present invention, the dynamic data concerning forces, accelerations, and the like, of a scriber, such as a pen, are collected during the process of cursive writing. The criteria of stability, smoothness, and synchronization of the cursive writing motion of the individual are returned by the system as quantifying measures of the neurological function of that individual. In the practice of the present invention, a cursive handwriting pattern is considered as a sequence of ballistic strokes. Replication of a pattern may be generated from a single, high-level memory representation, acting as a motor program. The displacement of the scriber is a result of a natural double integration of the acceleration variations in time. As such, the scriber displacement does not represent the high frequency part of the initial signal spectrum suppressed by the integration process, and it cannot be expected to obtain critical first-hand information about fine motor control by measuring second-hand effects. That is why the method of the present invention is based on analysis of accelerations and forces as the source of complete and substantial information about handwriting dynamics. This approach allows detection of deterioration in an individual""s cursive handwriting, even in cases of small, normally invisible deviations in handwriting.
The method takes as an input handwriting dynamic signals gathered on a millisecond time scale, and returns, as an output, very precise measurements of stability, smoothness, and synchronization of the handwriting movements. The method of the present invention focuses on how a person writes as opposed to how writing looks. The system translates the writing dynamics of acceleration and pressure into complex signals, with each signal represented by as many as 1000 data points. Then, as a key part of the signal processing, any two writing signals, which represent repeated realizations of the same handwriting samples are compared in order to establish statistical similarity of the signals. The samples might be, any sequence of cursive writing, including signatures and hand-drawn symbols.
The method uses a new approach for Correlation Function Analysis applied to behavioral signals, such as handwriting dynamics. The behavioral signals of cursive writing are not stable by nature, and relate to xe2x80x9cnon-stationaryxe2x80x9d signals, and subsequently are intractable for classical correlation function analysis. One key feature of the present invention relies on the discovery that those behavioral signals are stationary for a very short period of time, about 100 milliseconds, the order of a biological quantum, and that the quanta are intermittent with random length intervals. Those stationary signal segments may be considered as realizations of the preprocessed brain program of the individual implemented by nerve impulse propagation and by muscle contraction. The random intervals between those segments are understood to represent variations in the nerve stream propagation and in muscle contraction between the biological quanta. In other words, statistical stationarity has been found to have a granular character, and the information about fine motor skill is represented by the assembly of xe2x80x9cgranulesxe2x80x9d, as well as by the manner in which the intervals between granules occur.
In order to maintain the fine motor control assessments of the present invention with high accuracy, a xe2x80x9cwindow-shadowxe2x80x9d mechanism, as detailed below, is used to discriminate granules against the intervals between them. Then the method determines the level of cross-correlation between quasi-stationary signals consisting of those granules glued with one another, as well as analyzes the behavior of the intervals in order to characterize stability, smoothness and synchronization of handwriting movements.
The present invention also teaches a biometric machine method for instantaneous assessment of the fine motor control of a human individual through analysis of the handwriting dynamics of that individual using a scriber. A computer collects the dynamic data concerning forces, accelerations, and other properties of the scriber during the process of cursive writing. The criteria of stability, smoothness and synchronization of the hand motion of the writer are analyzed by a computer as quantifying measures of the neurological function of the human individual. The method processes behavioral random signals by the application of the Correlation Function Analysis to the handwriting dynamic signals. In addition, the frequency analysis of the signals is performed using Fourier analysis.
In one machine method embodiment, the system of the present invention includes a small, lightweight instrumented scriber, such as a pen, connected to a computer, power supply, and an analog-to-digital converter of handwriting samples made on any writing surface. In a second embodiment, the system comprises a digital tablet having an active grid surface and a non-instrumented scriber, with the handwriting samples collected on the active surface area of the grid. The method is simple and not invasive. The analysis takes less than a minute to return the numerical criteria scores and graphical-displays showing stability of the handwriting strokes and the characteristics of the phase-distortions in reproducing cursive samples.
The machine method can be implemented as a stationary system with use of a desktop computer and as a portable system with the use of a portable computer. The present invention can be used for early diagnostics of Parkinson""s disease and other neurological disorders, for monitoring individuals with known neurological disorders in order to determine that persons response to treatment, for expediting clinical trials of neurological pharmaceuticals, for monitoring people with alcohol/drug abuse problems, for the detection of fine motor control deterioration as a result of exposure to toxic environmental materials, or for testing the efficacy of countermeasures to improve impaired fine motor control functions.
The system of the present invention makes assessment of fine motor control, through measurement and analysis of handwriting dynamics representing both ballistic patterns, such as signature, and cognitively controlled cursive patterns of cursive patterns. Such ballistic and cognitively controlled cursive patterns have been found to provide precious and previously unavailable information about the role and interplay of cerebellum and cerebrum in adaptive control and cognition. It is in this vein that the method of the present invention, by measuring handwriting disturbances, provides useful information for the complex mechanisms of fine motor control, and provides information concerning human individuals with neurological disorders, in cases of drug/alcohol impairment and environmental distress. The method also allows evaluation of the effectiveness of therapeutic treatment of the neurological patients, and the design of adequate training protocols for learning disability problems.
These and other objects of the present invention will become apparent to those skilled in the art from the following detailed description, showing the contemplated invention as herein described, and more particularly defined by the appended claims, it being understood that changes to the precise embodiments of the herein disclosed invention are meant to be included as coming within the scope of the claims, except as they may be precluded by the prior art.